{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/NeuromatchAcademy/course-content/blob/master/tutorials/W1D2_ModelingPractice/W1D2_Tutorial1.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Neuromatch Academy: Week 1, Day 2, Tutorial \n",
    "# Modeling Practice: Framing the question\n",
    "\n",
    "__Content creators:__ Marius 't Hart, Megan Peters, Paul Schrater, Gunnar Blohm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# Tutorial objectives\n",
    "Yesterday you gained some understanding of what models can buy us in neuroscience. But how do you build a model? Today, we will try to clarify the process of computational modeling, by thinking through the logic of modeling based on your project ideas.\n",
    "\n",
    "We assume that you have a general idea of a project in mind, i.e. a preliminary question, and/or phenomenon you would like to understand. You should have started developing a project idea yesterday with [this brainstorming demo](https://youtu.be/H6rSlZzlrgQ). Maybe you have a goal in mind. We will now work through the 4 first steps of modeling ([Blohm et al., 2019](https://doi.org/10.1523/ENEURO.0352-19.2019)): \n",
    "\n",
    "**Framing the question**\n",
    "\n",
    "1. finding a phenomenon and a question to ask about it\n",
    "2. understanding the state of the art\n",
    "3. determining the basic ingredients\n",
    "4. formulating specific, mathematically defined hypotheses\n",
    "\n",
    "The remaining steps 5-10 will be covered in a second notebook that you can consult throughout the modeling process when you work on your projects.\n",
    "\n",
    "**Importantly**, we will guide you through Steps 1-4 today. After you do more work on projects, you likely have to revite some or all of these steps *before* you should move on the the remaining steps of modeling. \n",
    "\n",
    "**Note**: there will be no coding today. It's important that you think through the different steps of this how-to-model tutorial to maximize your chance of succeeding in your group projects.\n",
    "\n",
    "**Think! Sections**: All activities you should perform are labeled with **Think!**. These are discussion based exercises and can be found in the Table of Content on the left side of the notebook. Make sure you complete all within a section before moving on!\n",
    "\n",
    "Enjoy!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "#Step 1: Finding a phenomenon and a question to ask about it\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "execution": {
     "iopub.execute_input": "2021-05-30T16:40:34.229789Z",
     "iopub.status.busy": "2021-05-30T16:40:34.229011Z",
     "iopub.status.idle": "2021-05-30T16:40:34.348933Z",
     "shell.execute_reply": "2021-05-30T16:40:34.348306Z"
    }
   },
   "outputs": [],
   "source": [
    "#@title Video 1: Asking a question\n",
    "from IPython.display import YouTubeVideo\n",
    "video = YouTubeVideo(id=\"Prf_Tc9UNp0\", width=854, height=480, fs=1)\n",
    "print(\"Video available at https://youtu.be/\" + video.id)\n",
    "video"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Think! 1: Asking your own question\n",
    "You should already have a project idea from your brainstorming yesterday. **Write down the phenomenon, question and goal(s) if you have them.** \n",
    "\n",
    "As a reminder, here is what you should discuss and write down:\n",
    "\n",
    "* What exact aspect of data needs to be modelled?\n",
    "  * Answer this question clearly and precisely!\n",
    "Otherwise you will get lost (almost guaranteed)\n",
    "  * Write everything down!\n",
    "  * Also identify aspects of data that you do not want to address (yet)\n",
    "* Define the model evaluation method!\n",
    "  * How will you know your model is good?\n",
    "  * E.g. comparison to specific data (quantitative method of comparison?)\n",
    "* Think of an experiment that could test your model\n",
    "  * You essentially want your model to interface with this experiment, i.e. you want to simulate this experiment\n",
    "\n",
    "You can find interesting questions by looking for phenomena that differ from your expectations. In *what* way does it differ? *How* could that be explained (starting to think about mechanistic questions and structural hypotheses)? *Why* could it be the way it is? What experiment could you design to investigate this phenomenon? What kind of data would you need?\n",
    "\n",
    ">### Avoid these common pitfalls\n",
    "* Question is too general\n",
    "  * Remember: science advances one small step at the time. Get the small step right…\n",
    "* Precise aspect of phenomenon you want to model is unclear\n",
    "  * You fail to ask a meaningful question\n",
    "* You have already chosen a toolkit\n",
    "  * This will prevent you from thinking deeply about the best way to answer your scientific question\n",
    "* You don’t have a clear goal\n",
    "  * What do you want to get out the model?\n",
    "* You don’t have a potential experiment in mind\n",
    "  * This will help concretize your objectives and think through the logic behind your goal\n",
    "\n",
    "**This should take no more than 30 minutes.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "#Step 2: Understanding the state of the art\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here you will do a literature review (**to be done AFTER this tutorial!**). For the projects, do not spend too much time on this. A thorough literature review should take weeks or months depending on your prior knowledge of the field...\n",
    "\n",
    "The important thing for your project here is not to exhaustively survey the literatire but rather to learn the process of modeling. 1-2 days of digging into the literature should be enough!\n",
    "\n",
    "**Here is what you should get out of it**:\n",
    "* Survey the literature\n",
    "  * What’s known?\n",
    "  * What has already been done?\n",
    "  * Previous models as a starting point?\n",
    "  * What hypotheses have been emitted in the field?\n",
    "  * Are there any alternative / complementary models?\n",
    "* What skill sets are required?\n",
    "  * Do I need learn something before I can start?\n",
    "  * Ensures that no important aspect is missed\n",
    "* Provides specific data sets / alternative models for comparison\n",
    "\n",
    "**Do this AFTER the tutorial**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "#Step 3: Determining the basic ingredients\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "execution": {
     "iopub.execute_input": "2021-05-30T16:40:34.355748Z",
     "iopub.status.busy": "2021-05-30T16:40:34.355143Z",
     "iopub.status.idle": "2021-05-30T16:40:34.479989Z",
     "shell.execute_reply": "2021-05-30T16:40:34.480499Z"
    }
   },
   "outputs": [],
   "source": [
    "#@title Video 2: Determining basic ingredients\n",
    "from IPython.display import YouTubeVideo\n",
    "video = YouTubeVideo(id=\"hfntEL8TVvY\", width=854, height=480, fs=1)\n",
    "print(\"Video available at https://youtu.be/\" + video.id)\n",
    "video"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Think! 3: Determine your basic ingredients.\n",
    "\n",
    "Determine the basic ingredients. This will allow you to think deeper about what your model will need. It's a crucial step before you can formulate hypotheses because you first need to understand what the model will need to contain. There are 2 aspects you want to think about:\n",
    "\n",
    "1. What parameters / variables are needed in the model?\n",
    "  * Constants?\n",
    "  * Do they change over space, time, conditions…?\n",
    "  * What details can be omitted?\n",
    "  * Constraints, initial conditions?\n",
    "  * Model inputs / outputs?\n",
    "2. Variables needed to describe the process to be modelled?\n",
    "  * Brainstorming!\n",
    "  * What can be observed / measured?  latent variables?    \n",
    "  * Where do these variables come from?\n",
    "  * Do any abstract concepts need to be instantiated as variables?\n",
    "    * E.g. value, utility, uncertainty, cost, salience, goals, strategy, plant, dynamics\n",
    "    * Instantiate them so that they relate to potential measurements! \n",
    "\n",
    "This is a step where your prior knowledge and intuition is tested. You want to end up with an inventory of *specific* concepts and/or interactions that need to be instantiated. \n",
    "\n",
    ">### Make sure to avoid the following pitfalls\n",
    "* I’m experienced, I don’t need to think about ingredients anymore\n",
    "  * Or so you think…\n",
    "* I can’t think of any ingredients\n",
    "  * Think about the potential experiment. What are your stimuli? What parameters? What would you control? What do you measure?\n",
    "* I have all inputs and outputs\n",
    "  * Good! But what will link them? Thinking about that will start shaping your model and hypotheses\n",
    "* I can’t think of any links (= mechanisms)\n",
    "  * You will acquire a library of potential mechanisms as you keep modeling and learning\n",
    "  * But the literature will often give you hints through hypotheses\n",
    "  * If you still can't think of links, then maybe you're missing ingredients?\n",
    "\n",
    "The hardest part is Step 1. Once that is properly set up, all other should be easier. **BUT**: often you think that Step 1 is done only to figure out in later steps (anywhere really) that you were not as clear on your question and goal than you thought. Revisiting Step 1 is frequent necessity. Don't feel bad about it. You can revisit Step 1 later; for now, let's move on to the nest step.\n",
    "\n",
    "**This should take approximately 30 min.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "#Step 4: Formulating specific, mathematically defined hypotheses\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "execution": {
     "iopub.execute_input": "2021-05-30T16:40:34.485771Z",
     "iopub.status.busy": "2021-05-30T16:40:34.484701Z",
     "iopub.status.idle": "2021-05-30T16:40:34.566969Z",
     "shell.execute_reply": "2021-05-30T16:40:34.567500Z"
    }
   },
   "outputs": [],
   "source": [
    "#@title Video 3: Formulating a hypothesis\n",
    "from IPython.display import YouTubeVideo\n",
    "video = YouTubeVideo(id=\"Vc3g1XajLlc\", width=854, height=480, fs=1)\n",
    "print(\"Video available at https://youtu.be/\" + video.id)\n",
    "video"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Think! 4: Formulating your hypothesis\n",
    "\n",
    "Once you have your question and goal lines up, you have done a literature review (let's assume for now) and you have thought about ingredients needed for your model, you're now ready to start thinking about *specific* hypotheses.\n",
    "\n",
    "Formulating hypotheses really consists of two consecutive steps:\n",
    "1. You think about the hypotheses in words by relating ingredients identified in Step 3\n",
    "  * What is the model mechanism expected to do? \n",
    "  * How are different parameters expected to influence model results?\n",
    "\n",
    "2. You then express these hypotheses in mathematical language by giving the ingredients identified in Step 3 specific variable names. \n",
    "  * Be explicit, e.g. y(t)=f(x(t),k) but z(t) doesn’t influence y\n",
    "\n",
    "\n",
    "There are also \"structural hypotheses\" that make assumptions on what model components you hypothesize will be crucial to capture the phenomenon at hand. \n",
    "\n",
    "**Important**: Formulating the hypotheses is the last step before starting to model. This step determines the model approach and ingredients. It provides a more detailed description of the question / goal from Step 1. The more precise the hypotheses, the easier the model will be to justify. \n",
    "\n",
    ">### To succeed here, avoid the following pitfalls\n",
    "* I don’t need hypotheses, I will just play around with the model\n",
    "  * Hypotheses help determine and specify goals. You can (and should) still play…\n",
    "* My hypotheses don’t match my question (or vice versa)\n",
    "  * This is a normal part of the process!\n",
    "  * You need to loop back to Step 1 and revisit your question / phenomenon / goals\n",
    "* I can’t write down a math hypothesis\n",
    "  * Often that means you lack ingredients and/or clarity on the hypothesis\n",
    "  * OR: you have a “structural” hypothesis, i.e. you expect a certain model component to be crucial in explaining the phenomenon / answering the question\n",
    "\n",
    "**This step should take about 30 min**.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "\n",
    "\n",
    "**We now have everything we need to actually start modelling!**\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "execution": {
     "iopub.execute_input": "2021-05-30T16:40:34.573373Z",
     "iopub.status.busy": "2021-05-30T16:40:34.572688Z",
     "iopub.status.idle": "2021-05-30T16:40:34.661772Z",
     "shell.execute_reply": "2021-05-30T16:40:34.661134Z"
    }
   },
   "outputs": [],
   "source": [
    "#@title Video 4: Next step!\n",
    "from IPython.display import YouTubeVideo\n",
    "video = YouTubeVideo(id=\"bDqwjCYhyAg\", width=854, height=480, fs=1)\n",
    "print(\"Video available at https://youtu.be/\" + video.id)\n",
    "video"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "#Summary\n",
    "\n",
    "In this tutorial, we worked through some steps of the process of modeling. \n",
    "\n",
    "- We defined a phenomenon and formulated a question (step 1)\n",
    "- We collected information the state-of-the-art about the topic (step 2)\n",
    "- We determined the basic ingredients (step 3), and used these to formulate a specific mathematically defined hypothesis (step 4)\n",
    "\n",
    "You are now in a position that you could start modeling without getting lost. But remember: you might have to work through steps 1-4 again after doing a literature review and/or if there were other pitfalls you identified along the way (which is totally normal).\n",
    "\n",
    "[In the next tutorial](https://colab.research.google.com/drive/10689_o9Aea0PImoVhZ26IVxfK9sRHRuM?usp=sharing) we will continue with the steps 5-10 to guide you through the implementation and completion stages of the projects. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "#Reading\n",
    "Blohm G, Kording KP, Schrater PR (2020). _A How-to-Model Guide for Neuroscience_. eNeuro, 7(1) ENEURO.0352-19.2019. https://doi.org/10.1523/ENEURO.0352-19.2019 \n",
    "\n",
    "Kording KP, Blohm G, Schrater P, Kay K (2020). _Appreciating the variety of goals in computational neuroscience_. Neurons, Behavior, Data Analysis, and Theory 3(6). https://nbdt.scholasticahq.com/article/16723-appreciating-the-variety-of-goals-in-computational-neuroscience\n",
    "\n",
    "Schrater PR, Peters MK, Kording KP, Blohm G (2019). _Modeling in Neuroscience as a Decision Process_. OSF pre-print. https://osf.io/w56vt/"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "include_colab_link": true,
   "name": "W1D2_Tutorial1",
   "provenance": [],
   "toc_visible": true
  },
  "kernel": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
